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THE UNDER-OVER-UNDER THEOREM 

FRANCIS SCHEID, Boston University 

An experimenter has made three measurements of a quantity Q , at time 
h, h, and k. Conforming to currently accepted ritual of experimentation he plots 
three points. What they look like is shown in Figure 1. 

From discussions with expert theorists in his field our man knows that the 
points should fall on a straight line. The relationship between Q and t should be 
linear. The formula Q — Mt+B is indicated. Unfortunately, theory does not say 
which straight line. It does not predict the values of M and B. Indeed the pur- 
pose of the present experiment was the obtaining of this missing information. 
Many experiments are similarly inspired. One of Albert Einstein's accomplish- 
ments as a young man (before relativity) was the determination of an unknown 
coefficient in a theoretically derived formula for Brownian motion, the random 
motion of suspended particles in a supporting medium (dust in quiet air for 
example). So, theory often leaves little problems of this sort in its wake. 



Fig. 1 


Apparently, however, experiment also allows room for opinion. The three 
points in the diagram do not fall on a straight line. Taking his data home with 
him for the evening our man, in a moment of weakness, wishes he had made only 
two measurements. Then realizing full well that such thoughts are idle he de- 
cides to make the best of what he has, applies a transparent ruler to the diagram, 
and after a few moments of indecision draws his choice. It passes under, over, 
and under the three points in turn (Fig. 2). From his diagram he estimates 

M — rise/run = .5, B = —.2 

and instantly wonders just how good these values are. Has he made the best 
choice of straight line? 

He is not by any means the first human being to face this question. Indeed, 
various “best” solutions have been suggested over the passing years. What 
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many believe to be the best of the best appears first in the work of the nineteenth- 
century mathematician Cebysev (alias Tchebysheff, alias Tchebycheff, alias 
Chebyshev). Chebyshev argues as follows. Choose any line whatever. It will 
pass over or under the three points by amounts hi, h 2 , h z to be called errors (posi- 
tive for over, negative for under). A direct hit simply makes one error zero. Let 
H be the largest error size, the worst miss, for the line chosen. For every line 
there is such an H. Then, of all possible straight lines, the best according to 
Chebyshev is the one whose H , the biggest miss, is smallest. The maximum error 
size must be minimized. It is a M INMAX problem. 



Now let’s take the analytic view. The equation y = Mt+B represents a 
straight line. With M and B unspecified it could be almost any line. We are to 
determine M and B by applying the Chebyshev idea of bestness to the experi- 
mental data. The values of Y may then be thought of as corrections of the data 
values Q. Let the data be quite generally 

t\ t 2 t Z 

Qi Q 2 Qz 

so that the errors become 

(1) hi = yi — Qi, hi = y 2 — Q 2 , h z = y z — Q z , 

with 

(2) yi = Mli + B, y 2 = Mt 2 + B, y 3 = Mt z + B. 

A few lines of algebra will now convince the deepest skeptic that 

(^3 — ^)yi — (h — h)y 2 + (^2 — h)yz = 0. 

(Just substitute yi, y 2 , y% from above.) For brevity this is rewritten as 

(3) ($iyi — /3 2 y2 + fi z y z = 0, 
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where Pi = t z — t 2 , $2 = h — h, /3 3 = t 2 — ti. It may be assumed that h<t 2 <t z so that 
all three jS’s are positive. 

We now raise the question “Is there a line for which 
hi = h, h 2 = — h, h z = h, 

making the three errors of equal size and alternating sign ?” If h is positive this 
line goes over-under-over the three points, while if h is negative the line goes 
under-over-under. 

This question may seem thoroughly irrelevant and immaterial. But in fash- 
ionable courtroom language “it will be connected up.” Tradition in mathemati- 
cal journalism frowns on following the winding paths taken by our mathematical 
ancestors to their own discoveries. To continue the metaphor, it is considered 
wasteful of time to re-enact the crime. Our question happens to be a short cut, 
though this is perhaps not at once obvious. 

If a line having the suggested property does exist it is easy to deduce what 
h must be. For then 

yi = Qi + h , y 2 = Q 2 - h , yz = Q* + h , 

and substituting into (3), 

fii(h + Qi) — /3 2 ( — h + Q 2 ) + fiz(h + Q 3 ) = 0, 
leading quickly to 

... 7 /3l(?l — /3 2 (?2 + /?3@3 

( 4 ) h = , 

/ 5 i + /?2 + /?3 

so that only one h is possible. Our line must therefore pass through the three 
points 

t\ t 2 tz 

Qi + h Q 2 — h Qz + h 

with h given by (4). 

But this is slightly severe. It is common knowledge that passing through 
just two specified points is all that one should ask of a straight line. It narrows 
the field of competition to exactly one candidate. That there is a line through the 
above three points is therefore newsworthy, if true. That it is true may be dis- 
covered by comparing the slopes of P\P 2 and P 2 P Z (Pi, P 2 , and P z being the three 
points, taken left to right). It requires a little persistence, but using (4) and the 
fact that j8i+j33 = j32 the equality of these slopes emerges and settles the issue 
beyond further doubt. 

The answer to our question is thus, “Yes, there is a line, exactly one, which 
makes the errors hi, h 2) h z of equal size and alternating sign.” 

Now comes the connecting up. The line just found will be called the Cheby- 
shev line, for it is the best line in Chebyshev’s sense. The proof is easy. As indi- 
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cated, h , — h, h are the errors for the Chebyshev line. Let fa , fa, fa be the errors 
for any other line . Then using (4), (1) and (3) in succession 

. Pi(yi ~ fa) — p 2 (y 2 — h 2 ) + pz{yz — fa) 
h = — 

£i + £2 + /?3 

(5) — __ H ~ (ftjh ~~ P%h 2 + /?3^3) 

/?1 + /?2 + $3 

/?i^i — fizfa + p$hz 

Pi + p2 + /?3 

But if H is, as suggested before, the maximum of | fa\ , j fa | , | fa\ , then because all 
jS’s are positive the right side of (5) is certainly increased if hi, fa , fa are replaced 
by H , —H,H respectively. This means 


( 6 ) 


/Si# + P2B + fizH 

Pi + @2 + P* 


= E. 


Thus the maximum error size, | h \ , of the Chebyshev line is no larger than the 
maximum, H , for any other line. This proves that | h\ is the required minmax. 
By example, if the three points of our original diagram are as they seem to be 


t: 0 12 

Q: 0 0 1 

then Pi = 2-1 = 1, p 2 = 2-0 = 2, fo=l-0 = l, and by (4), 

h= (i)(Q) ~ (2) (0) + (1)(1) _ 

1 + 2+1 


T- 


The Chebyshev line thus passes through 

t: 0 12 


and so has the equation y = ^t — \ as may easily be verified. This line passes 
under, over and under the three points missing each by the same amount, and 
yields the predictions M=. 5, B = 25, which are certainly very close to the 

results our experimenter obtained by eye. As a matter of fact the average per- 
son faced with the same problem will draw by eye a line fairly close to the 
Chebyshev line. 

Can there be two best lines? In other words, can some other line have the 
same maximum error as the Chebyshev line? In this case H=\h\, and the 
equality sign holds in (6). At this point a few moments of quiet thought are more 
useful than a thousand words, and should eventually bring conviction that the 
substitution which carries (5) into (6) cannot preserve equality unless fa, fa , fa 
are of size \ h\ and of alternating sign. But it is these features which led us to 
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the three points through which the Chebyshev line passes. Surely there are not 
two straight lines through these three points. The uniqueness of the best line 
is a fact. 

Let’s suppose next that our experimenter is not entirely satisfied with his 
first effort. Returning to his apparatus next morning he discovers that he has for- 
gotten to shut it off. This brings mixed emotions, since it is an expensive affair 
and he has not been authorized to let it go on so long. However, it would be 
criminal to waste such a golden opportunity, and accordingly he obtains two 
additional readings before his conscience forces him to shut down. Adding these 
two readings to yesterday’s plot he has the picture (Fig. 3) before him. (We 
may assume that his readings are from a counter, so that all have integer 
values.) It is now plain that his old line is too high. It passes over both new 
points, missing the first by quite a bit. Having no reason to suppose this par- 



ticular reading any less worthy than the other four, he once again applies the 
transparent ruler and draws a new line. Figure 4 shows what the picture now 
looks like. From the new line he makes the estimates 

M = rise/run = 2.75/7 = .39, B = — .1, 

which differ, of course, from his previous estimates but which he suspects are 
somewhat closer to the fact. 

And what does Chebyshev have to say about this more complicated prob- 
lem? Which line is really the best line? Chebyshev’s answer is the same as before. 
It is the line for which the maximum error size H is the smallest. Becoming 
analytic once more let the data be rather generally 

^1 * * * tn 
(?!•*• Qn- 

In our example we take the values suggested by the diagram 
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t: 0 12 6 7 

Q : 0 0 1 2 3 . 

For any straight line y = Mt+B the ordinates yi and errors hi are again given by 
equations just like (1) and (2), there simply being more of them. Again H is 
the maximum of the quantities \hi\ and the best line for the given points, ac- 
cording to Chebyshev, is the line whose H is the smallest. Now, how can this 
best line be found? The following exchange method provides a very effective 
answer to this question. It proceeds in four basic steps, and does yield the best 
line as we shall see. 

Step one. Choose any three of the data points . (A set of three data points will 
be called a triple . This step simply selects an initial triple . It will be changed in 
Step four.) Proceed to Step two. 



Step two. Find the Chebyshev line for this triple as illustrated above . The value 
h for this line will , of course , be calculated in the process . Proceed to Step three . 

In illustration of these first two steps let us choose the initial triple 

t: 0 12 

Q: 0 0 1 

consisting of the first three of the five points. This is, of course, the triple occur- 
ring in our original example. The Chebyshev line for this triple was found to be 
y = \t — \ with h——\. This line passes under, over and under the first three of 
our five points with equal error sizes. But as noted earlier its errors at the re- 
maining two points suggest a new choice of line. The exchange method makes 
this new choice as follows. 

Step three. Calculate the errors at all remaining points for the Chebyshev 
line just found. Call the largest of these hi values {in absolute value) H. If \h\ =H 
the search is over. The Chebyshev line for the chosen triple is also the best line for the 
entire point set. {We shall prove this in a moment .) If \h\ <H proceed to Step four. 
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In our example we find 

h = — i = h, 'I 12 = l = — h, h z = — J = h } h± = f, h& = |. 

Thus i?=f and since this exceeds \h\ we are not finished and proceed as in- 
structed to 

Step four. This is the exchange step . Choose a new triple as follows. Add to 
the old triple a data point at which the greatest error size H occurs. Then discard 
one of the former points , in such a way that the remaining three have errors of alter- 
nating sign. (A moment's practice will show that this is always possible .) Then re- 
turn , with the new triple , to Step two. 

In the example we must now include the fourth point and eliminate the first. 
On the new triple 

t: 12 6 

Q : 0 12 

the errors & 2 , hz, h 4 found a moment ago do have the required alternation of sign. 
With this new triple we return as instructed to Step two : 

Again we are to find a Chebyshev line. The computation follows the pattern 
of our first effort: 

^ = 6-2 = 4, fi % = 6 - 1 = 5, 0* = 2-1 = 1, 

h (4)(0) - (5)(1) + (1)(2) 3 

4+5+1 10 ' 

The Chebyshev line must pass through 

12 6 

'V* 3 7 2 3 

y * TF TIT 1 0 

and so has the equation 

0) J = it “ Ttr* 

This again completes Step two. Repeating Step three as instructed we find 
hi = — xtr, h?, — -yq = A, hz = A = h, h± = y 3 Q- = A, hf> = — 

This time H—-^ so that | h\ = H and the job is done. The Chebyshev line (7) 

on the triple at /= (1, 2, 6) is the best line for the set of five points with which 
we began. Its maximum error, H=^q, is the smallest possible for any line. This 
will now be proved in the general case. Let us stop, however, to record the 
latest predictions of M and B , namely, 

M = .4, B = - .1, 

which should certainly tend to boost sales of transparent rulers. 
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There are two open questions. The exchange method can come to a halt 
only in Step three, and then only if no error exceeds \h\. How do we know this 
condition is ever satisfied? (Conceivably we could be computing forever.) That 
is the first question. The second is, of course, assuming the condition is satisfied 
at some stage, why is the last Chebyshev line also the best line in the Chebyshev 
sense for the whole point set? 

To answer the first question, recall that after any particular exchange the 
old Chebyshev line has errors of size \h \ , | h\ , H on the new triple. Also recall 
that \h\ <H (or we would have stopped) and that these errors alternate in 
sign. The Chebyshev line for this new triple is then found. Call its errors on the 
triple A*, — A*, A*. Returning to (5) now, with the old Chebyshev line playing 
the role of “any other line,” we have 

_ . Plfa — filfa + fizfa 

hr — ; 

/5l + /?2 + i^3 

where fa, fa, fa are the numbers h , In, H with alternating sign. Because of this 
alternation of sign all three terms in the numerator of this fraction have the 
same sign, so that 

U*| _ +ftl *| +&# 

i#l + /?2 + £3 

if we assume that the error H is at the third point, just to be specific. (It really 
makes no difference in which position it goes.) In any event, | h*\ >\h\ because 
| h\ <H. The new Chebyshev line has a greater error size on its triple than the 
old one had (on its own triple) . This result now gives excellent service. If it comes 
as a surprise, look at it this way. The old line in our example gave excellent 
service (h = l) on its own triple, but poor service (H= |) elsewhere. The new line 
gave good service (h = -£o) on its own triple, and just as good service ( H=h ) 
on the other points also. 

We can now prove that the exchange method must come to a stop sometime. 
For, there are only so many triples. And no triple is ever chosen twice, since as just 
proved the h values increase steadily. At some stage the condition | h\ = H will be 
satisfied. 

The second question is almost as easily answered as the first. For the last 
Chebyshev line, and the h value of its triple, the maximum error size on the 
whole point set is H= | h\ , (or we would have proceeded by another exchange to 
still another triple). If fa, • • • , h n are the errors for any other line, then by (6) 

| h\ ^max | hi\ , where hi is restricted to the three points of the last triple. But 
then certainly \ h\ ^max | fa\ , where fa is unrestricted, for including the remain- 
ing points can only make the right side even bigger. Thus the maximum error, 
H= \h\, of this final Chebyshev line is the smallest maximum error of all. It 
is the minmax. The best line in the Chebyshev sense is one which has errors of 
equal size and alternating sign on a properly chosen triple. This is what we have 
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light-heartedly designated as the under-over-under theorem. Taking one 
moment more, let us again show that the best line is unique. If a second line 
had maximum error (on the entire point set) equal to that of the final Chebyshev 
line, which is | h \ , then on the final triple its maximum error would certainly be 
no greater. But it could also be no less, for, as we saw earlier, no line can outdo 
the Chebyshev line on its own triple. So on the final triple both lines would have 
the same maximum error. Our earlier proof of uniqueness now applies. The two 
lines must be the same. 

As a final example we may suppose that our laboratory scientist has received 
a new research grant, which restores his fiscal respectability and permits him to 
repeat his experiment on a grander scale. The results are the following experi- 
menters delight. 


t: 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 


Q- 

0 

1 

1 

2 

1 

3 

2 

2 

3 

5 

3 

4 

5 

4 

5 


t: 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 

26 

27 

28 

29 

30 

Q- 

6 

6 

5 

7 

6 

8 

7 

7 

8 

7 

9 

11 

10 

12 

11 

13 


With this wealth of expensive information he abandons his transparent ruler 
and consults higher authority on best-fitting straight lines. The search for the 
correct triple and corresponding Chebyshev line is on. There are 31 points to 
choose among. First graders may soon be learning that the number of possible 
triples in this accumulation is C(31, 3) or 31!/(28! 3!) =4495. Finding the cor- 
rect triple in this haystack appears at first glance to be a task well left to the 
arithmetical wizardry of automatic computing machines. Fortunately the ex- 
change method does have the repetitious nature so easy to explain to (program 
for) such machines. And fortunately too the method wastes very little time on 
inconsequential triples. It hastens to its target like a hungry lion. The IBM 650 
at Boston University took less than five minutes and only three exchanges, start- 
ing from the horrible initial triple at /= (0, 1, 2), to produce the Chebyshev line 
(shown in Fig. 5). 

y = .38095/ - .28571 

on the triple at /=( 9, 24, 30). The value of h for this line is 1.85714 and the 
maximum error outside the triple is only 1.61905 so that truly this is the best 
line in Chebyshev’s sense. It yields the predictions 

M = .38095, B = — .28571 

for the long-sought coefficients. These may be compared with the cruder values 
found earlier, and presumably will serve the purpose for the present. If the for- 
ward march of science leads to the suspicion that these predictions are grossly 
in error, then perhaps a better experimental method can be devised, or hopefully 
more funds will become available. But for the given data the consensus of 
current mathematical literates is that the Chebyshev line is the best. 
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The exchange method has rooted out the correct triple so brilliantly in this 
example that the use of high-speed machines seems more a luxury than a neces- 
sity. It is interesting to watch the progress of the computation through the 
points included in the successive triples and the corresponding h and H values. 


Triple at t = 

( 0 , 1 , 2 ) 

(0, 1, 24) 

(1.24.30) -1 

(9.24.30) -1 


h 

H 

25000 

5.25000 

35417 

-3.89583 

75862 

-2.44828 

85714 

-1.85714 


Note that in this case no unwanted point is ever brought into the triple. Three 
points are needed, three exchanges suffice. Note also the steady increase of \ h\ 9 
as forecast by theory. 



Though the machine is really unnecessary for the above calculations, the 
existence of modern computers has led during the past decade to a rapid spread 
of the Chebyshev method to more complicated problems of approximation. Not 
only may straight lines be fitted by the minmax criterion, but a variety of more 
sophisticated curves as well. For these the search may be much more laborious 
(perhaps for a correct octuple). For various reasons such problems of approxi- 
mation are fundamental in modern computation, and it is easy to believe that 
the machine becomes man’s indispensable colleague. 

Problem. Show that the best line of our last example {shown in Fig . S) makes 
direct hits at t = 6 and at t = 27. There are no other direct hits. 



